Vehicle and control method thereof

ABSTRACT

Disclosed are a vehicle and a control method thereof that may allow an occupant as well as a driver to conveniently use a speech recognition system by utilizing a microphone of a mobile device as a source of speech input, that may utilize a microphone of a mobile device as a source of noise collection, when a driver uses a speech recognition system. A vehicle may include a microphone; a speaker; a communication module configured to communicate with a mobile device; and a controller. In a first mode, the communication module is configured to receive a voice signal of a user from the mobile device, and in the first mode, the controller is configured to perform noise cancellation on the user&#39;s voice signal received from the mobile device, based on an audio signal input to the microphone.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 U.S.C. § 119to Korean Patent Application No. 10-2022-0017057, filed on Feb. 9, 2022,in the Korean Intellectual Property Office, the entire contents of whichare incorporated herein by reference.

BACKGROUND 1. Technical Field

The present disclosure relates to a vehicle and a control method thereofthat may provide a service corresponding to a user's speech.

2. Description of the Related Art

A speech recognition system is capable of identifying what is intendedby a user from a user's speech and providing a service corresponding tothe identified user intention.

A speech recognition system is linked to a specific device to controlthe device and provide specific information according to a userintention.

Because a vehicle is equipped with a speech recognition system, a usermay control the vehicle, obtain desired information or be provided withdesired services simply through dialogues with the vehicle.

However, because a speech recognition system in a vehicle is designed tobe used mainly by a driver, an occupant in the car has limitations inusing the speech recognition system.

SUMMARY

An aspect of the disclosure provides a vehicle and a control methodthereof that may allow an occupant as well as a driver to convenientlyuse a speech recognition system by utilizing a microphone of a mobiledevice as a source of speech input.

An aspect of the disclosure provides a vehicle and a control methodthereof that may utilize a microphone of a mobile device as a source ofnoise collection when a driver uses a speech recognition system.

Additional aspects of the disclosure are set forth in part in thedescription which follows and, in part, should be apparent from thedescription or may be learned by practice of the disclosure.

According to an aspect of the present disclosure, a vehicle includes: amicrophone provided inside the vehicle; a speaker provided inside thevehicle; a communication module configured to communicate with a mobiledevice located inside the vehicle; and a controller. In particular, in afirst mode, the communication module is configured to receive a voicesignal of a user from the mobile device, and in the first mode, thecontroller is configured to perform noise cancellation on the user'svoice signal received from the mobile device, based on an audio signalinput to the microphone.

In another embodiment, in a second mode, the communication module isconfigured to receive an audio signal from the mobile device, and in thesecond mode, the controller is configured to perform noise cancellationon a voice signal of the user input to the microphone, based on theaudio signal received from the mobile device.

In one embodiment, the controller is configured to control the speakerto output a test sound in a registration mode for registering the mobiledevice.

In another embodiment, in the registration mode, the controller isconfigured to calculate a delay of the mobile device based on a testsound signal, received from the mobile device after controlling thespeaker to output the test sound, and map and store the calculated delayto the mobile device.

In one embodiment, in the first mode, the controller is configured tosynchronize the audio signal input to the microphone and the user'svoice signal received from the mobile device, based on the delay mappedto the mobile device.

In the second mode, the controller is configured to synchronize theuser's voice signal input to the microphone and the audio signalreceived from the mobile device, based on the delay mapped to the mobiledevice.

In one embodiment, the controller is configured to operate in the firstmode, when a speech recognition trigger signal is input to the mobiledevice, and operate in the second mode, when the speech recognitiontrigger signal is input to the vehicle.

In one embodiment, the communication module is configured to receive avoice signal of an occupant of the vehicle from the mobile device in thefirst mode, and the microphone is configured to receive a speech of adriver of the vehicle in the second mode.

In one embodiment, the communication module is configured to transmitthe user's voice signal where the noise cancellation is performed to aspeech recognition system.

When a command corresponding to the user's voice signal received fromthe mobile device is allowable, the controller is configured to generatea control signal for performing the command, and when the commandcorresponding to the user's voice signal received from the mobile deviceis not allowable, generate a guide message.

According to an aspect of the present disclosure, a control method of avehicle includes: determining a first mode or a second mode related to aspeech recognition; when the first mode is in operation, receiving avoice signal of a user from a mobile device located inside the vehicle;and when the first mode is in operation, performing noise cancellationon the user's voice signal received from the mobile device, based on anaudio signal input to a microphone provided in the vehicle.

In one embodiment, the control method further includes: when the secondmode is in operation, receiving an audio signal from the mobile device;and when the second mode is in operation, performing noise cancellationon a voice signal of the user input to the microphone, based on theaudio signal received from the mobile device.

The control method further includes registering the mobile device in thevehicle.

In one embodiment, the registering of the mobile device in the vehicleincludes controlling a speaker, provided in the vehicle, to output atest sound.

In one embodiment, the registering of the mobile device in the vehiclefurther includes: calculating a delay of the mobile device based on atest sound signal, received from the mobile device after controlling thespeaker to output the test sound; and mapping and storing the calculateddelay to the mobile device.

In one embodiment, the performing of noise cancellation includes, whenthe first mode is in operation, synchronizing the audio signal input tothe microphone and the user's voice signal received from the mobiledevice, based on the delay mapped to the mobile device.

In one embodiment, the performing of noise cancellation includes, whenthe second mode is in operation, synchronizing the user's voice signalinput to the microphone and the audio signal received from the mobiledevice, based on the delay mapped to the mobile device.

In one embodiment, the determining of the first mode or the second moderelated to the speech recognition includes: determining as the firstmode when a speech recognition trigger signal is input to the mobiledevice and determining as the second mode when the speech recognitiontrigger signal is input to the vehicle.

In one embodiment, when the first mode is in operation, the user's voicesignal received from the mobile device includes a voice signal of anoccupant of the vehicle, and when the second mode is in operation, theuser's voice signal input to the microphone includes a voice signal of adriver of the vehicle.

In one embodiment, the control method further includes: when a commandcorresponding to the user's voice signal received from the mobile deviceis allowable, generating a control signal for performing the command;and when the command corresponding to the user's voice signal receivedfrom the mobile device is not allowable, generating a guide message.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects of the disclosure should become apparent andmore readily appreciated from the following description of embodiments,taken in conjunction with the accompanying drawings of which:

FIG. 1 is a block diagram illustrating a speech recognition systemconnected to a vehicle according to an embodiment;

FIG. 2 is a block diagram illustrating a vehicle according to anembodiment, and FIG. 3 is a diagram schematically illustrating arelationship between a speech recognition system and a vehicle accordingto an embodiment;

FIG. 4 is a block diagram illustrating a mobile device providing amicrophone to a vehicle according to an embodiment;

FIG. 5 is a diagram illustrating types of signals exchanged between avehicle according to an embodiment and a mobile device connectedthereto;

FIG. 6 is a flowchart illustrating operations of registering a mobiledevice in a vehicle, in a control method of the vehicle according to anembodiment;

FIG. 7 is a diagram illustrating operations of a controller of a vehicleaccording to an embodiment;

FIG. 8 is a flowchart illustrating operations of measuring a delay, in acontrol method of a vehicle according to an embodiment;

FIG. 9 is a diagram illustrating an example of a time difference betweena test sound output from a vehicle according to an embodiment and asignal transmitted to the vehicle from a mobile device;

FIG. 10 is a diagram illustrating an example where a microphone of avehicle according to an embodiment is used for speech input and amicrophone of a mobile device is used for noise collection;

FIG. 11 is a diagram illustrating an example where a microphone of avehicle according to an embodiment is used for noise collection and amicrophone of a mobile device is used for speech input;

FIG. 12 is a flowchart illustrating operations where a vehicle uses anaudio signal provided from a mobile device, in a control method of thevehicle according to an embodiment;

FIG. 13 is a diagram illustrating an example of a screen displayed on amobile device, when speech recognition is triggered through the mobiledevice;

FIG. 14 is a flowchart illustrating operations of processing a speechcommand differently depending on an object to which the speech commandis input, in a control method of a vehicle according to an embodiment;and

FIG. 15 is a diagram illustrating an example of a guide signal outputwhen a speech command is not performable, in a vehicle according to anembodiment.

DETAILED DESCRIPTION

It should be understood that embodiments may be replaced with variousequivalents and modifications at the time of the disclosure.

Terminologies used herein are for the purpose of describing particularembodiments only and are not intended to limit the disclosure. It shouldbe understood that the singular forms are intended to include the pluralforms as well, unless the context clearly dictates otherwise. It shouldbe further understood that the terms “include,” “comprise” and/or “have”when used in this specification, specify the presence of statedfeatures, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,integers, steps, operations, elements, components, and/or groupsthereof.

Further, the terms such as “part,” “device,” “block,” “member,”“module,” and the like, may refer to a unit for processing at least onefunction or act. For example, the terms may refer to at least processprocessed by at least one hardware, such as field-programmable gatearray (FPGA)/application specific integrated circuit (ASIC), softwarestored in memories or processors.

It should be understood that, although the terms “first,” “second,” andthe like, may be used herein to describe various elements, theseelements should not be limited by these terms. When a component, device,element, or the like of the present disclosure is described as having apurpose or performing an operation, function, or the like, thecomponent, device, or element should be considered herein as being“configured to” meet that purpose or to perform that operation orfunction.

Reference numerals used for method steps are just used for convenienceof explanation, but the reference numerals are not used to limit anorder of the steps. Thus, unless the context clearly dictates otherwise,the written order may be practiced otherwise.

The term “at least one” used herein includes any and all combinations ofthe associated listed items. For example, it should be understood thatthe term “at least one of a, b, or c” may include only a, only b, onlyc, both a and b, both a and c, both b and c, or all of a, b and c.

Embodiments can be stored in the form of a recording medium storingcomputer-executable instructions. The instructions may be stored in theform of a program code, and when executed by a processor, theinstructions may perform operations of the disclosed embodiments. Therecording medium may be implemented as a computer-readable recordingmedium.

The computer-readable recording medium includes all kinds of recordingmedia in which instructions which may be decoded by a computer arestored of, for example, a read only memory (ROM), random access memory(RAM), magnetic tapes, magnetic disks, flash memories, optical recordingmedium, and the like.

Embodiments of a vehicle and a control method thereof should bedescribed in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating a speech recognition systemconnected to a vehicle according to an embodiment.

In an embodiment, a speech recognition system connected to a vehicle mayrefer to a speech recognition system that may be used when called fromthe vehicle.

Referring to FIG. 1 , a speech recognition system 100 includes a speechprocessing module 10 that extracts information required to perform acontrol intended by a user from a voice signal of the user, and acontroller 130 that generates a control signal for performing thecontrol intended by the user.

The speech processing module 10 may include a speech recognition module110 converting a user's speech into text and a natural languageunderstanding module 120 determining a user intention corresponding tothe text.

The speech recognition module 110 may be implemented with a speech totext (STT) engine and perform conversion into text by applying a speechrecognition algorithm to the user's speech.

For example, the speech recognition module 110 may extract featurevectors from a user's speech by applying a feature vector extractionmethod such as a cepstrum, a linear predictive coefficient (LPC), a Melfrequency cepstral coefficient (MFCC), a filter bank energy, or thelike.

Also, a recognition result may be obtained by comparing extractedfeature vectors and trained reference patterns. An acoustic model formodeling and comparing signal characteristics of voice or a languagemodel for modeling a linguistic order of recognition vocabulary such aswords or syllables may be used.

In addition, the speech recognition module 110 may convert the user'sspeech into the text based on learning where deep learning or machinelearning is applied. According to an embodiment, a way of converting theuser's speech into the text by the speech recognition module 110 is notlimited thereto, and a variety of speech recognition techniques may beapplied to convert the user's speech into the text.

The natural language understanding module 120 may apply a naturallanguage understanding (NLU) technique to determine user intentionincluded in the text. Accordingly, the natural language understandingmodule 120 may include an NLU engine that determines the user intentionby applying the NLU technique to an input sentence. The text output bythe speech recognition module 110 may be a sentence input to the naturallanguage understanding module 120.

For instance, the natural language understanding module 120 mayrecognize an entity name from the input sentence. The entity name is aproper noun such as a name of an individual person, place, organization,time, day, currency, and the like. Named-entity recognition is foridentifying an entity name in a sentence and classifying a type of theidentified entity. A keyword may be extracted from a sentence throughnamed-entity recognition to understand the meaning of the sentence.

Also, the natural language understanding module 120 may determine adomain from the input sentence. The domain may be for identifying asubject of the user's speech. For example, domains representing varioussubjects such as a vehicle control, schedule, information about weatheror traffic conditions, text transmission, navigation, and the like, maybe determined based on the input sentence.

In addition, the natural language understanding module 120 may analyze aspeech act of the input sentence. Speech act analysis is for analyzingan intention of speech, such as whether the user asks a question, makesa request, responds, or simply expresses the user's emotions.

The natural language understanding module 120 may determine an intentand an entity required to perform the intent, based on the domain,entity name, speech act extracted from the input sentence. For example,when the input sentence is “turn on the air conditioner,” the domain maybe [vehicle control] and the intent may be [turn on, air conditioner].[Turn on] may be an action, [air conditioner] may be a target, and theentity required to perform control corresponding to such intent may be[temperature, air volume].

The controller 130 may perform processing on the result of the speechrecognition and natural language understanding and output a resultprocessing signal to a user terminal or an external server, in order toprovide a service corresponding to a user intention. For example, thecontroller 130 may generate and output a control signal for performing acontrol corresponding to an intent extracted from a user's speech.

The user terminal may serve as a gateway between the user and the speechrecognition system 100. The user terminal may be a mobile deviceprovided with an input/output interface such as a microphone, a speaker,a display, and the like, or be a vehicle itself. When the user terminalis a mobile device, the vehicle and the mobile device may be connectedto each other through a wireless communication such as Bluetooth™, orthrough a cable connection.

For example, when a service corresponding to a user intention is avehicle-related control, the controller 130 may generate a controlsignal for performing the corresponding control and transmit the controlsignal to the user terminal.

Alternatively, when a service corresponding to a user intention isprovision of specific information, the controller 130 may search for thespecific information and transmit the retrieved information to the userterminal. Information retrieval may be performed by an external server,when required.

Alternatively, when a service corresponding to a user intention isprovision of specific content, the controller 130 may request to anexternal server providing the corresponding content.

Alternatively, when a service corresponding to a user intention issimply continuation of a dialogue, the controller 130 may generate aresponse to a user's speech and output the response by voice.

The above-described speech recognition system 100 may be implementedwith at least one memory storing a program performing the aforementionedoperations and at least one processor implementing a stored program.

The constituent components of the speech recognition system 100illustrated in FIG. 1 are divided based on their operation or function,and all or a portion of the constituent components may share the memoryor processor. The speech recognition module 110, the natural languageunderstanding module 120, and the controller 130 are not necessarilyphysically separated from each other.

FIG. 2 is a block diagram illustrating a vehicle according to anembodiment, and FIG. 3 is a diagram schematically illustrating arelationship between a vehicle and a speech recognition system accordingto an embodiment.

Referring to FIG. 2 , a vehicle 2 includes: a microphone 210 to which auser's speech is input; a speaker 220 outputting a sound required toprovide a service desired by a user; a display 230 displaying an imagerequired to provide a service desired by the user; a communicationmodule 240 performing communication with an external device; and acontroller 250 controlling the above-described constituent componentsand other constituent components of the vehicle 2.

The microphone 210 may be provided inside the vehicle 2 to receive auser's speech. The user that inputs the user's speech to the microphone210 of the vehicle 2 may be a driver. The microphone 210 may be providedon a steering wheel, a center console, a headliner, or a rear-viewmirror, and the like, to receive a speech from the driver.

Various audios generated around the microphone 210 may be input to themicrophone 210 in addition to the user's speech. The microphone 210 mayoutput an audio signal corresponding to the audio input to themicrophone 210, and the output audio signal may be processed by thecontroller 250 or transmitted to the speech recognition system 100through the communication module 240.

The vehicle 2 may also include an inputter 260 for manually receiving auser command in addition to the microphone 210. The inputter 260 mayinclude an inputter provided as a jog shuttle or a button, in an areawhere an audio, video, navigation (AVN) is provided on a center console,in an area where a gearbox is provided, or on a steering wheel.

Also, to receive a control command related to passenger seats, theinputter 260 may include an inputter provided on each door of thevehicle 2, and an inputter provided on a front armrest or a reararmrest.

Further, the inputter 260 may include a touchpad like a touchscreen bybeing provided integrally with the display 230.

The display 230 may include an AVN display provided on a center consoleof the vehicle 2, a cluster display, or a head-up display (HUD).Alternatively, the display 230 may include a rear seat display providedon a back of the front seat's headrest so that a rear occupant may seethe rear seat display. When the vehicle 2 is a multi-seater vehicle, thedisplay 230 may include a display mounted on a headliner of the vehicle2.

The display 230 may be provided anywhere as long as users inside thevehicle 2 may see the display 230, and the position or the number ofdisplays 230 are not limited.

The communication module 240 may transmit and receive a signal withother devices by using at least one of various wireless communicationmethods, such as Bluetooth™, fourth-generation wireless (4G),fifth-generation wireless (5G), wireless-fidelity (Wi-Fi), and the like.Alternatively, the communication module 240 may transmit and receiveinformation with other devices through a cable connected to a universalserial bus (USB) terminal, an auxiliary port (AUX) terminal, and thelike.

Also, the communication module 240 may be provided with two or morecommunication interfaces that support different communication methods totransmit and receive information with two or more other devices.

For example, the communication module 240 may communicate with a mobiledevice located inside the vehicle 2 through a Bluetooth™communication,thereby receiving information (user images, user speech, contactnumbers, schedules, and the like) obtained by or stored in the mobiledevice. Also, the communication module 240 may communicate with a server1 through a 5G communication method, thereby transmitting a user'sspeech and receiving a signal required to provide a service desired bythe user. In addition, the communication module 240 may transmit andreceive a signal with the server 1 through a mobile device connected tothe vehicle 2.

Further, the vehicle 2 may include: a navigation device for routeguidance; an air conditioning device for adjusting an indoortemperature; a window adjustment device for opening/closing vehiclewindows; a seat heating device for heating seats; a seat adjustmentdevice for adjusting a position, height, angle of a seat; a lightingdevice for adjusting an indoor illuminance level; and the like.

The aforementioned constituent components are for providing conveniencefunctions related to the vehicle 2, and a portion of the constituentcomponents may be omitted depending on models or options. Also, otherconstituent components may be further included in the vehicle 2 inaddition to the above constituent components. Configurations related todriving of the vehicle 2 are well-known, and thus a description thereofis omitted herein.

The controller 250 may turn on or off the microphone 210, process andstore a speech input to the microphone 210 and transmit to anotherdevice through the communication module 240.

Also, the controller 250 may control the display 230 to display an imageand control the speaker 220 to output a sound.

In addition, the controller 250 may perform various controls related tothe vehicle 2. For example, the controller 250 may control at least oneof: the navigation device; the air conditioning device; the windowadjustment device; the seat heating device; the seat adjustment device;or the lighting device, according to a user command input through theinputter 260 or the microphone 210.

The controller 250 may include at least one memory storing a programperforming the aforementioned operations or operations to be describedbelow and at least one processor implementing a stored program.

Referring to FIG. 3 , the speech recognition system 100 may be providedin the server 1. Accordingly, a user's speech input to the vehicle 2 maybe transmitted to a communication module 140 of the server 1, and when avoice signal is processed by the speech recognition system 100 providedin the server 1, the communication module 140 may transmit a processingresult to the vehicle 2 again.

The communication module 140 may transmit and receive a signal withother devices by using at least one of various wireless communicationmethods, such as Bluetooth™, 4G, 5G, Wi-Fi, and the like.

A portion of the constituent components of the speech recognition system100 may be provided in the vehicle 2, and the other constituentcomponents may be provided in the server 1.

For example, the speech recognition module 110 may be provided in thevehicle 2, and the natural language understanding module 120 and thecontroller 130 may be provided in the server 1.

As another example, the speech recognition module 110 and the controller130 may be provided in the vehicle 2 and the natural languageunderstanding module 120 may be provided in the server 1, or the speechrecognition module 110 and the natural language understanding module 120may be provided in the server 1 and the controller 130 may be providedin the vehicle 2.

As another example, the speech recognition system 100 may be provided inthe vehicle 2.

Although all or a portion of the constituent components of the speechrecognition system 100 may be provided in the vehicle 2 as describedabove, an example where the speech recognition system 100 is provided inthe server 1, as shown in FIG. 3 , is described in the below-describedembodiment.

As described above, the microphone 210 to which a user's speech is inputis provided in the vehicle 2, and an audio signal output from themicrophone 210 is transmitted to the speech recognition system 100 andis used to determine a user intention. Also, according to an embodiment,by using a microphone of a mobile device located inside the vehicle 2 asa microphone for receiving a speech of an occupant other than a driver,the occupant may also conveniently use the speech recognition system 100connected to the vehicle 2. Thus, because a separate microphone forreceiving an occupant's speech is not required for the vehicle 2,convenience of the vehicle may be improved, and cost reduction may beachieved.

FIG. 4 is a block diagram illustrating a mobile device providing amicrophone to a vehicle according to an embodiment.

Referring to FIG. 4 , a mobile device 3 includes: a microphone 310; aspeaker 320; a display 330; a communication module 340; a controller350; and an inputter 360.

The mobile device 3 may be a smartphone, a tablet, a laptop, or awearable device such as a smart watch or smart glasses.

A variety of audios including a user's voice may be input to themicrophone 310 and the microphone 310 may output an audio signalcorresponding to the input audio. The audio signal output from themicrophone 310 may be used variously. The audio signal may betransmitted to the other party's device during a call and stored in amemory of the mobile device 3 during recording. Also, the audio signalmay be transmitted to a speech recognition system connected to themobile device 30.

In addition, when the mobile device 3 is connected to the vehicle 2according to an embodiment, the audio signal output from the microphone310 may be transmitted to the vehicle 2.

The display 330 may display screens required to perform variousfunctions of the mobile device 30. For example, when the mobile device 3is connected to the vehicle 2 according to an embodiment, a variety ofscreens accompanying execution of an application for connection with thevehicle 2 may be displayed.

The communication module 340 may transmit and receive a signal withother devices by using at least one of various wireless communicationmethods, such as Bluetooth™, 4G, 5G, Wi-Fi, and the like. Alternatively,the communication module 340 may transmit and receive a signal withother devices through a cable connected to a USB terminal, an AUXterminal, and the like.

For example, the communication module 340 may be connected to thevehicle 2 through a Bluetooth™ communication. When connected to thevehicle 2, the communication module 340 may transmit informationrequested from the vehicle 2 to the vehicle 2.

For instance, the communication module 340 may transmit information suchas contact numbers, call history, or text history stored in the mobiledevice 3 to the vehicle 2. Alternatively, as is described below, thecommunication module 340 may transmit the audio signal output from themicrophone 310 to the vehicle 2.

A user may also input a desired command to the mobile device 3 by voice.The mobile device 3 may be connected to a speech recognition system, andthus the speech recognition system connected to the mobile device 3 maybe different from or the same as the speech recognition system 100connected to the vehicle 2.

When a user inputs a user's speech to the microphone 310 to control themobile device 3, the audio signal output from the microphone 310 may betransmitted to the speech recognition system through the communicationmodule 340.

The transmitted audio signal may be processed by the speech recognitionsystem, and when a control signal for performing a control correspondingto a user intention is received from the speech recognition system, themobile device 3 may control the constituent components according to thereceived control signal.

The mobile device 3 may include an inputter 360 for manually receiving acommand other than a voice command through manipulation, like thevehicle 2. The inputter 360 may include a device like a button providedon an exterior of the mobile device 3, or a touchpad like a touchscreenprovided integrally with the display 330.

FIG. 5 is a diagram illustrating types of signals exchanged between avehicle according to an embodiment and a mobile device connectedthereto.

After the vehicle 2 according to an embodiment and the mobile device 3located in the vehicle 2 are connected to each other by wire orwirelessly, when operations to be described below are performed, amicrophone of the mobile device 3 may be used as a microphone forreceiving an occupant's speech or collecting noise, as shown in FIG. 5 .Accordingly, the mobile device 3 may provide the vehicle 2 with an audiosignal including a voice signal or a noise signal.

Also, the vehicle 2 may transmit to the mobile device 3, a controlsignal as to which microphone of the vehicle 2 or the mobile device 3 isused for speech input or noise collection.

As is described below, an audio signal may be transmitted or receivedthrough an audio channel between the vehicle 2 and the mobile device 3,and a control signal may be transmitted or received through a controlchannel between the vehicle 2 and the mobile device 3.

A user may install a dedicated application for sharing a microphone withthe vehicle 2 in the mobile device 3, in order to use the microphone ofthe mobile device 3 as described above. Among the operations to bedescribed below, operations performed by the mobile device 3 may beperformed by executing the installed application.

FIG. 6 is a flowchart illustrating operations of registering a mobiledevice in a vehicle, in a control method of a vehicle according to anembodiment. FIG. 7 is a diagram illustrating operations of a controllerof a vehicle according to an embodiment.

A control method of a vehicle according to an embodiment refers to acontrol method of the vehicle 2 according to an embodiment. Accordingly,the control method of a vehicle according to an embodiment may beperformed by the vehicle 2, and a description of operations of thevehicle 2 are equally applicable to embodiments of the control method ofa vehicle, even when not specifically described below.

Referring to FIG. 6 , it is confirmed whether the vehicle 2 and themobile device 3 are connected (1100). The vehicle 2 and the mobiledevice 3 may be connected to each other through a wireless communicationmethod such as Bluetooth™, or through a wired communication method usinga cable.

When the mobile device 3 is connected (Yes in operation 1100), it isconfirmed whether the mobile device 3 is a new device or an alreadyregistered device (1200).

When the mobile device 3 is a new device (Yes in operation 1200), aprocess for registering the newly connected mobile device 3 isperformed, i.e., entering a registration mode.

First, it may be confirmed whether the connected mobile device 3 has anavailable microphone (1300). For example, it may be confirmed whetherthe mobile device 3 supports a hands-free profile, and when a hands-freeprofile is supported, it may be determined that an available microphoneis provided in the mobile device 3 (Yes in operation 1300).

A delay of the mobile device 3 is measured (1400). In order for an audioinput to the mobile device 3 to be used in the vehicle 2, a process oftransmitting an audio from the mobile device 3 to the vehicle 2 isrequired, and a delay occurs in the above transmission process.

Accordingly, when the mobile device 3 is newly connected, the vehicle 2may measure a delay between the mobile device 3 and the vehicle 2 (1400)and store the measured delay in the corresponding mobile device 3(1500). When an audio signal, provided from the mobile device 3, isused, the stored delay may be used to synchronize with an audio signaloutput from the microphone 210 of the vehicle 2.

When the delay is stored and the microphone 310 of the mobile device 3may be used for the vehicle 2, a speech recognition standby mode isentered (1600).

Referring to FIG. 7 , the above-described operations may be performed bythe controller 250 of the vehicle 2. The controller 250 of the vehicle 2may be referred to as a head unit (HU) and include at least one memoryand at least one processor.

As described above with reference to FIG. 6 , the controller 250 mayconfirm whether the microphone 310 is present each time the new mobiledevice 3 is connected to the vehicle 2, and may register the new devicein the vehicle 2 by mapping information about whether the microphone 310is present to an identification (ID) of each mobile device 3 and storingin a memory.

Also, the controller 250 may measure a delay for the newly registeredmobile device 3, map a delay value to the ID of each mobile device 3 andstore in the memory.

In addition, the controller 250 may cancel noise by performingpre-processing on an audio signal output from the microphone 210 of thevehicle 2 or the microphone 310 of the mobile device 3, as describedbelow.

FIG. 8 is a flowchart illustrating operations of measuring a delay, in acontrol method of a vehicle according to an embodiment. FIG. 9 is adiagram illustrating an example of a time difference between a testsound output from a vehicle according to an embodiment and a signaltransmitted to the vehicle from a mobile device.

Referring to FIG. 8 , in order to measure the delay of the newlyconnected mobile device 3 (1400), the controller 250 of the vehicle 2outputs a test sound through the speaker 220 (1410). The test sound isoutput to be input to the microphone 310 of the mobile device 3.Accordingly, before outputting the test sound, the vehicle 2 maytransmit a control signal for opening the microphone 310 to the mobiledevice 3.

The test sound output from the speaker 220 of the vehicle 2 may be inputto the open microphone 310 of the mobile device 3 (2010).

The mobile device 3 transmits the test sound input to the microphone 310to the vehicle 2 through the communication module 340 (2020).

The communication module 240 of the vehicle 2 receives the test soundtransmitted from the mobile device 3 (1420). A communication method usedby the communication module 240 of the vehicle 2 and the communicationmodule 340 of the mobile device 3 for transmission and reception of thetest sound is the same as that used for transmission and reception of anaudio signal later.

The controller 250 of the vehicle 2 calculates a delay based on thereceived test sound (1430). As shown in FIG. 9 , a time differencebetween a time point when the vehicle 2 outputs the test sound throughthe speaker 220 and a time point when the test sound is received againfrom the mobile device 3, i.e., a delay T(d), exists.

As described above, the controller 250 stores the calculated delay valueT(d) in the memory, and when an audio signal is received from the mobiledevice 3 later, the stored delay value may be used for synchronizationwith an audio signal output from the microphone 210 of the vehicle 2.

FIG. 10 is a diagram illustrating an example where a microphone of avehicle according to an embodiment is used for speech input and amicrophone of a mobile device is used for noise collection. FIG. 11 is adiagram illustrating an example where a microphone of a vehicleaccording to an embodiment is used for noise collection and a microphoneof a mobile device is used for speech input.

Referring to an example of FIG. 10 , when a speech recognition istriggered by an occupant which is a user of the mobile device 3, themicrophone 310 of the mobile device 3 connected to the vehicle 2 may beused for speech input, and the microphone 210 of the vehicle 2 may beused for noise collection.

The occupant may input a speech command to the microphone 310 of themobile device 3, and while the occupant inputs the occupant's speechcommand to the microphone 310 of the mobile device 3, the microphone 210of the vehicle 2 may collect noise.

The speech command input to the mobile device 3 may be transmitted tothe vehicle 2, and the vehicle 2 may cancel noise from the transmittedspeech command using the collected noise.

Referring to an example of FIG. 11 , when a speech recognition istriggered by a driver, the microphone 210 of the vehicle 2 may be usedfor speech input, and the occupant's mobile device 3 connected to thevehicle 2 may be used for noise collection.

The driver may input a speech command for controlling the vehicle 2 tothe microphone 210 of the vehicle 2, and while the driver inputs thedriver's speech command to the microphone 210 of the vehicle 2, theoccupant's mobile device 3 connected to the vehicle 2 may collect noise.

The collected noise is transmitted to the vehicle 2 and may be used tocancel noise from the driver's speech input to the vehicle 2.

The mobile device 3 connected to the vehicle 2 may be a mobile device ofthe occupant or a mobile device of the driver. For example, when thedriver is driving with the mobile device 3 placed in a passenger seat,the microphone 210 of the vehicle 2 may be used for speech input, andthe mobile device 3 may be used for noise collection.

According to the above-described embodiments, a microphone which is notused for speech input among the microphone 210 of the vehicle 2 and themicrophone 310 of the mobile device 3 may be used for noise collection,thereby effectively cancelling noise without adding an additionalmicrophone.

Operations of using the microphone 210 of the vehicle 2 and themicrophone 310 of the mobile device 3 for speech recognition or noisecollection are described in detail.

FIG. 12 is a flowchart illustrating operations where a vehicle uses anaudio signal provided from a mobile device, in a control method ofvehicle according to an embodiment. FIG. 13 is a diagram illustrating anexample of a screen displayed on a mobile device, when speechrecognition is triggered through the mobile device.

Referring to FIG. 12 , when a speech recognition is triggered (Yes inoperation 2100), it is determined whether the speech recognition istriggered by the vehicle (Yes in operation 2200) or the mobile device(No in operation 2200). When the speech recognition is triggered by themobile device, a first mode where the microphone 310 of the mobiledevice 3 is used for speech recognition and the microphone 210 of thevehicle 2 is used for noise collection is entered. When the speechrecognition is triggered by the vehicle, a second mode where themicrophone 310 of the mobile device 3 is used for noise collection andthe microphone 210 of the vehicle 2 is used for speech input is entered.

For example, when the speech recognition is triggered through a push totalk (PTT) button provided in the vehicle 2, or when a wake-up wordwhich is a call word for calling the speech recognition system 100 isinput to the microphone 210 of the vehicle 2, it may be determined thatthe speech recognition is triggered by the vehicle 2.

A dedicated application installed in the mobile device 3 may be executedto trigger the speech recognition through the mobile device 3. Whenrunning the application, as shown in an example of FIG. 13 , a screenfor asking whether to use a microphone may be displayed, and when a‘yes’ button is selected, the microphone 310 may be open and the speechrecognition may be triggered through the mobile device 3. A speechrecognition trigger signal may be transmitted to the vehicle 2.

Alternatively, a user may utter a wake-up word for calling the speechrecognition system used in the mobile device 3. By uttering thecorresponding wake-up word, the microphone 310 of the mobile device 3may be open and a speech recognition trigger signal may be transmittedto the vehicle 2.

The controller 350 of the mobile device 3 may recognize that theutterance of the wake-up word while connected to the vehicle 2 is atrigger signal for transmitting an audio signal to the vehicle 2, not acall of the speech recognition system connected to the mobile device 3.Accordingly, the controller 350 may transmit the audio signal outputfrom the microphone 310 of the mobile device 3 to the vehicle 2 only,without transmitting to the speech recognition system connected to themobile device 3.

When the speech recognition is triggered by the vehicle 2 (Yes inoperation 2200), a guide signal may be transmitted to the mobile device3 (2300).

The vehicle 2 and the mobile device 3 may use an audio channel fortransmitting an audio signal and a control channel for mutual control.When the speech recognition is triggered by the vehicle 2, a noisesignal may be received from the mobile device 3 through the audiochannel, and when the speech recognition is triggered by the mobiledevice 3, a user's voice signal may be received from the mobile device 3through the audio channel.

When the speech recognition is triggered by the vehicle 2 (Yes inoperation 2200), the vehicle 2 transmits a guide signal for notifying astart of speech recognition to the mobile device through the controlchannel. The mobile device 3 may open the microphone 310 and collectnoise through the microphone 310.

The microphone 210 of the vehicle 2 receives a user's speech (2400). Theuser's speech may be a driver's speech.

When a speech input of the user ends (Yes in operation 2500), a guidesignal for notifying that the speech input has ended is transmitted tothe mobile device 3 (2600).

When receiving the guide signal, the mobile device 3 may close themicrophone 310 and transmit a collected noise signal to the vehicle 2.

The vehicle 2 may receive the noise signal from the mobile device 3(2700) and cancel noise from the user's voice signal based on thereceived noise signal (2800).

The controller 250 synchronizes a noise signal Noise(d), collected bythe microphone 310 of the mobile device 3, to a voice signal Voice(D),output from the microphone 210 of the vehicle 2, using a delay valueT(d) of the mobile device 3 stored in the memory. As given by Equation(1) below, by cancelling the synchronized noise signal Noise(ds) fromthe user's voice signal Voice(D), the pre-processed voice signal (Vcmd),i.e., the noise-cancelled voice signal, may be obtained.

Vcmd=Voice(D)−Noise(ds)   (1)

When the speech recognition is triggered by the mobile device 3 (No inoperation 2200), the vehicle 2 receives the guide signal for notifying astart of speech recognition from the mobile device 3 through the controlchannel (2910). The mobile device 3 may open the microphone 310 andreceive a user's speech through the microphone 310. The user's speechmay be an occupant's speech.

The vehicle 2 may open the microphone 210 and collect noise (2920).

When a speech input of the user ends, the mobile device 3 transmits theinput user's speech to the vehicle 2, and the vehicle 2 receives thevoice signal from the mobile device 3 (2930).

While the user's speech is input to the microphone 310 of the mobiledevice 3, the microphone 210 of the vehicle 2 collects noise.

The controller 250 of the vehicle 2 may cancel noise from the user'svoice signal based on the received noise signal (2800).

The controller 250 synchronizes a noise signal Noise(D), collected bythe microphone 210 of the vehicle 2, to a voice signal Voice(d),transmitted from the microphone 310 of the mobile device 3, using thedelay value T(d) of the mobile device 3 stored in the memory. As givenby Equation (2) below, by cancelling the synchronized noise signalNoise(Ds) from the user's voice signal Voice(d), the pre-processed voicesignal Vcmd, i.e., the noise-cancelled voice signal Vcmd, may beobtained.

Vcmd=Voice(d)−Noise(Ds)   (2)

FIG. 14 is a flowchart illustrating operations of processing a speechcommand differently depending on an object to which the speech commandis input, in a control method of a vehicle according to an embodiment.FIG. 15 is a diagram illustrating an example of a guide signal outputwhen a speech command is not performable, in a vehicle according to anembodiment.

Referring to FIG. 14 , when a voice signal where noise is cancelledaccording to the above-described operations is transmitted to the speechrecognition system 100 (3100), the speech recognition module 110 and thenatural language understanding module 120 of the speech recognitionsystem 100 may determine an intent corresponding to the transmittedvoice signal.

The speech recognition system 100 transmits a processing result of thevoice signal to the vehicle 2. The transmitted processing result of thevoice signal may include the intent corresponding to the voice signaland/or a control signal for performing a command indicated by thecorresponding intent.

The communication module 240 of the vehicle 2 receives the processingresult of the voice signal from the speech recognition system 100(3200).

The controller 250 of the vehicle 2 determines whether the commandcorresponding to the voice signal is allowable, based on whether thevoice signal is output from the microphone 310 of the mobile device 3 orthe microphone 210 of the vehicle 2 (3300).

The control method of the vehicle according to an embodiment may limittypes of commands that may be input through the mobile device 3, whichis based on an assumption that the command input through the mobiledevice 3 is made by an occupant, not a driver. For safety, only a driveris allowed to input a command for vehicle control, and an occupant isalso allowed to input a command for other controls except for thevehicle control.

The types of commands allowable according to a subject providing thevoice signal may be stored in the memory.

For example, a command that may be input through the microphone 210 ofthe vehicle 2 may include: a vehicle control; multimedia control;phone/text message; navigation; and the like. A command that may beinput through the microphone 30 of the mobile device 3 may includemultimedia control, phone/text message, and the like, except for thevehicle control, and include or exclude navigation according tosettings.

Alternatively, a driving-related control among the vehicle control maybe input only through the microphone 210 of the vehicle 2, and an airconditioning control, seat control of a passenger seat, lightingcontrol, and the like, which are irrelevant to driving, may be inputthrough the microphone 310 of the mobile device 3 as well.

Whether to allow a command according to a subject providing a voicesignal may be determined in units of domains or in units of individualcommands.

As a result of determination, when the command corresponding to thevoice signal is an allowable command (Yes in operation 3300), thecontroller 250 performs the command (3400). When the commandcorresponding to the voice signal is not allowable (No in operation3300), the controller 250 may generate and display a guide messageindicating that the requested command is not performable (3500), asshown in FIG. 15 . The guide message may be visually output through thedisplay 230, or audibly output through the speaker 220.

According to the above embodiments, an occupant as well as a driver mayconveniently use a speech recognition system without adding anadditional microphone, and noise cancellation may be effectivelyperformed, thereby improving a speech recognition rate.

As is apparent from the above, according to embodiments of thedisclosure, the vehicle and the control method thereof can allow anoccupant as well as a driver to conveniently use a speech recognitionsystem by utilizing a microphone of a mobile device as a source ofspeech input.

Also, when a driver uses a speech recognition system, a microphone of amobile device can be utilized as a source of noise collection, therebyreducing the number of microphones in a vehicle and costs.

Although embodiments have been described for illustrative purposes,those having ordinary skill in the art should appreciate that variousmodifications, additions and substitutions are possible, withoutdeparting from the scope and spirit of the disclosure. Therefore,embodiments have not been described for limiting purposes.

What is claimed is:
 1. A vehicle, comprising: a microphone providedinside the vehicle; a speaker provided inside the vehicle; acommunication module configured to communicate with a mobile devicelocated inside the vehicle; and a controller, wherein, in a first mode:the communication module is configured to receive a voice signal of auser from the mobile device, and the controller is configured to performnoise cancellation on the user's voice signal received from the mobiledevice, based on an audio signal input to the microphone.
 2. The vehicleof claim 1, wherein, in a second mode: the communication module isconfigured to receive an audio signal from the mobile device, and thecontroller is configured to perform noise cancellation on a voice signalof the user input to the microphone, based on the audio signal receivedfrom the mobile device.
 3. The vehicle of claim 2, wherein thecontroller is configured to control the speaker to output a test soundin a registration mode configured to register the mobile device.
 4. Thevehicle of claim 3, wherein, in the registration mode, the controller isconfigured to calculate a delay of the mobile device based on a testsound signal and map, wherein the test sound signal is received from themobile device after controlling the speaker to output the test sound,and the controller is configured to store the calculated delay to themobile device.
 5. The vehicle of claim 4, wherein, in the first mode,the controller is configured to synchronize the audio signal input tothe microphone and the user's voice signal received from the mobiledevice, based on the delay mapped to the mobile device.
 6. The vehicleof claim 4, wherein, in the second mode, the controller is configured tosynchronize the user's voice signal input to the microphone and theaudio signal received from the mobile device, based on the delay mappedto the mobile device.
 7. The vehicle of claim 2, wherein when a speechrecognition trigger signal is input to the mobile device, the controlleris configured to operate in the first mode, and when the speechrecognition trigger signal is input to the vehicle, the controller isconfigured to operate in the second mode.
 8. The vehicle of claim 2,wherein the communication module is configured to receive a voice signalof an occupant of the vehicle from the mobile device in the first mode,and the microphone is configured to receive a speech of a driver of thevehicle in the second mode.
 9. The vehicle of claim 2, wherein thecommunication module is configured to transmit the user's voice signalwhere the noise cancellation is performed to a speech recognitionsystem.
 10. The vehicle of claim 1, wherein, when a commandcorresponding to the user's voice signal received from the mobile deviceis allowable, the controller is configured to generate a control signalconfigured to perform the command, and wherein when the commandcorresponding to the user's voice signal received from the mobile deviceis not allowable, the controller is configured to generate a guidemessage.
 11. A control method of a vehicle, the control methodcomprising: determining a first mode or a second mode related to aspeech recognition; when the first mode is in operation, receiving avoice signal of a user from a mobile device located inside the vehicle;and when the first mode is in operation, performing noise cancellationon the user's voice signal received from the mobile device, based on anaudio signal input to a microphone provided in the vehicle.
 12. Thecontrol method of claim 11, further comprising: when the second mode isin operation, receiving an audio signal from the mobile device; and whenthe second mode is in operation, performing noise cancellation on avoice signal of the user input to the microphone, based on the audiosignal received from the mobile device.
 13. The control method of claim12, further comprising registering the mobile device in the vehicle. 14.The control method of claim 13, wherein registering the mobile device inthe vehicle comprises controlling a speaker, provided in the vehicle, tooutput a test sound.
 15. The control method of claim 14, whereinregistering the mobile device in the vehicle further comprises:calculating a delay of the mobile device based on a test sound signal,received from the mobile device after controlling the speaker to outputthe test sound; and mapping and storing the calculated delay to themobile device.
 16. The control method of claim 15, wherein performingnoise cancellation comprises: when the first mode is in operation,synchronizing the audio signal input to the microphone and the user'svoice signal received from the mobile device, based on the delay mappedto the mobile device.
 17. The control method of claim 15, whereinperforming noise cancellation comprises: when the second mode is inoperation, synchronizing the user's voice signal input to the microphoneand the audio signal received from the mobile device, based on the delaymapped to the mobile device.
 18. The control method of claim 12, whereindetermining the first mode or the second mode related to the speechrecognition comprises: determining as the first mode when a speechrecognition trigger signal is input to the mobile device, anddetermining as the second mode when the speech recognition triggersignal is input to the vehicle.
 19. The control method of claim 12,wherein, when the first mode is in operation, the user's voice signalreceived from the mobile device includes a voice signal of an occupantof the vehicle, and when the second mode is in operation, the user'svoice signal input to the microphone includes a voice signal of a driverof the vehicle.
 20. The control method of claim 11, further comprising:when a command corresponding to the user's voice signal received fromthe mobile device is allowable, generating a control signal configuredto perform the command; and when the command corresponding to the user'svoice signal received from the mobile device is not allowable,generating a guide message.